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Abstract 

Background: Despite endorsement by the Saskatchewan government to apply empirically-based approaches to youth 
drug prevention services in the province, programs are sometimes delivered prior to the establishment of evidence- 
informed goals and objectives. This paper shares the 'preparatory' outcomes of our team's program evaluation of the 
Prince Albert Parkland Health Region Mental Health and Addiction Services' Outreach Worker Service (OWS) in eight rural, 
community schools three years following its implementation. Before our independent evaluation team could assess 
whether expectations of the OWS were being met, we had to assist with establishing its overarching program goals and 
objectives and 'at-risk' student population, alongside its alliance with an empirically-informed theoretical framework. 

Methods: A mixed-methods approach was applied, beginning with in-depth focus groups with the OWS staff to identify 
the program's goals and objectives and targeted student population. These were supplemented with OWS and school 
administrator interviews and focus groups with school staff. Alignment with a theoretical focus was determined though a 
review of the OWS's work to date and explored in focus groups between our evaluation team and the OWS staff and 
validated with the school staff and OWS and school administration. 

Results: With improved understanding of the OWS's goals and objectives, our evaluation team and the OWS staff 
aligned the program with the Positive Youth Development theoretical evidence-base, emphasizing the program's 
universality, systems focus, strength base, and promotion of assets. Together we also gained clarity about the OWS's 
definition of and engagement with its 'at-risk' student population. 

Conclusions: It is important to draw on expert knowledge to develop youth drug prevention programming, but 
attention must also be paid to aligning professional health care services with a theoretically informed evidence-base for 
evaluation purposes. If time does not permit for the establishment of evidence-informed goals and objectives at the 
start-up of a program, obtaining insight and expertise from program personnel and school staff and administrators can 
bring the program to a point where this can still be achieved and theoretical linkages made after a program has been 
implemented. This is a necessary foundation for measuring an intervention's success. 

Keywords: Youth drug prevention program, Outreach worker service, Rural school, Positive youth development, Program 
evaluation, 40 developmental assets 
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Here is an example of a 17 year old with no social 
supports, we are trying to set him up, he has no 
modeling at home for budgets, cleanliness, tidiness, 
meals, etc. He has no family life, and stays in the 
computer lab after school until it closes. He is waiting 
when 1 arrive in the morning. He only eats bagels as 
apparently this is the only food in his house. So he 
started in the breakfast program at the school. His 
caregiver, Grandma, usually spends her time in the 
local bar. He is not a user [of substances] yet but he 
needs coping skills. - Prince Albert Parkland Health 
Region School Principal, Interview #4, 2012. 

Background 

It is well established that individuals who initiate the use of 
substances earlier in life have a greater likelihood that they 
will experience adverse consequences later in life, including 
addiction [1-3]. A combination of factors generally puts a 
youth, defined as 12-18 years of age, at-risk. Research has 
shown that the initiation of substance use is closely linked 
with social and environmental factors, whereas early initi- 
ation of use as well as abuse are highly linked to genetic 
and psychological determinants [2,4]. The determinants 
that place youth 'at-risk' of problematic substance use are 
generally categorized [5] as individual (e.g., age, gender, 
neurophysiological vulnerability) [1,6-11], interpersonal 
(e.g., family, peers, school), [2,9,12-16] and social and cul- 
tural/community (e.g., media portrayals, social norms, 
street involvement) [5,17-21]. 

Exposure to risk factors alone does not determine youth 
involvement in drug use and abuse. Increasingly research 
is showing the benefits of protective factors; community 
assets and individual resiliency have been identified to 
mitigate the influence of risk factors [12,16,17,22,23]. 
According to the work of Saeywc (2007), "[connectedness 
to school, positive relationships with caring adults within 
or outside of the family, and supportive peers seem to re- 
duce the likelihood of the distress and difficulties in cop- 
ing that lead to problem substance use" (18). A recent 
study by The McCreary Centre Society on building resili- 
ence in vulnerable youth concluded that "[p]ositive rela- 
tionships provide the most potent protective factors for 
vulnerable teens" [24] . Research has likewise identified the 
importance of resiliency in the residential treatment of 
youth for problematic substance use, such as at the 
Nimkee NupiGawagan Healing Centre for First Nations 
youth in Ontario [25]. 

Positive youth development (PYD) is a multidisciplinary 
theory that recognizes the strengths of youth and the 
communities in which they live [26-28] . The goal of PYD 
is to increase and sustain the positive, healthy develop- 
ment of young people [26], in particular adolescents, and 
has been applied to programs aimed at preventing sub- 
stance use [29] (see also Tebes et al, 2007 [30]). The 



reduction of high-risk behaviour is a core tenant of the 
PYD framework, recognizing that low risk negative 
behaviours are a part of healthy youth development 
[26,31,32]. PYD focusses on three key areas: being uni- 
versal (targets all youth rather than only 'at-risk' youth, 
with the understanding that all youth can benefit from 
strengthened environments; [26]), systems focussed 
(youth development is embedded within many inte- 
grated, interactive contexts, including family, school, 
community, society, culture and history [26,32]), and 
strength-based (views youth as resources to be nur- 
tured, encouraging the growth of inherent capacities 
for positive development and competence) [27,29,31]. 

A fourth element of PYD is the promotion of develop- 
mental assets, which synthesizes the theory's systems fo- 
cussed and strength-based areas into a structured and 
measurable evidence-base. Exemplified in the Search In- 
stitute's 40 Developmental Assets, the assets are aimed 
at increasing youth well-being and are often thought of 
as the 'building blocks' for healthy child and adolescent 
development [33,34]. This developmental asset approach 
is informed by the concepts of resiliency and protective 
factors, and is based on research pinpointing 20 external 
and 20 internal assets (see Table 1). These assets have 
been determined to: 1) prevent risky behaviours, includ- 
ing substance abuse and violence, 2) augment positive 
outcomes such as school success, and 3) support resili- 
ency in youth [35]. Researchers have found that the 
more developmental assets a young person has the more 
positive their development will be, or the less likely they 
will be to engage in risky behaviours [26,34,36]. For ex- 
ample, in a study assessing the utility of an instrument 
designed to measure substance use relative to develop- 
mental assets (i.e., the Search Institute Profiles of Stu- 
dent Life: Attitudes and Behaviors), it was found that for 
tobacco, alcohol and illicit drug use, on average 47% of 
youth in grades 6-12 with 0-10 developmental assets ei- 
ther used alcohol three or more times in the past month 
or got drunk once or more in the past 2 weeks, smoked 
one or more cigarettes every day or used chewing tobacco 
frequently, or used illicit drugs three or more times in the 
past year, whereas on average 2% of youth with 31-40 de- 
velopmental assets engaged in the same behaviours [35]. 

Positive youth development theory and the 40 develop- 
mental assets approach both recognize that specific youth 
populations may derive unique benefits from protective 
factors. It is well-established in the literature, for example, 
that violence and trauma, sexual and physical abuse, 
stigma and discrimination, and social attitudes and pres- 
sures differentially impact females and males [10,37]. The 
results of two meta-analyses relay that girls may benefit 
more than male participants from resistance skills training 
[38,39], programs that reduce social influences to use 
drugs [39], and altering perceived social norms [39]. There 



Table 1 40 Developmental assets for adolescents (12-18 years of age) [33] 



External assets 



Internal assets 



Support boundaries and expectations 



Empowerment constructive use of time 



Commitment to learning positive identity Positive values 



Family support: Family life provides high levels of love 
and support 

Positive family communication: Young person and 
her parent(s) communicate positively, and young person 
is willing to seek advice and counsel from parents. 

School boundaries: School provides clear rules 
and consequences. 



Parent involvement in schooling: Parent(s) are 
actively involved in helping the child succeed 
in school. 



Service to others: Young person serves in the 
community one hour or more a week. 

Other adult relationships: Young person 
receives support from three or more 
nonparent adults 

Safety: Young person feels safe at home, school 
and in the neighbourhood. 

Adult role models: Parent(s) and other adults 
model positive, responsible behaviour. 



Equality and social justice: Young person 
places high value on promoting equality 
and reducing hunger and poverty. 

Integrity: Young person acts on convictions 
and stands up for her 
or beliefs. 

Homework: Young person reports doing 
at least one hour of homework every 
school day. 

School engagement: Young person is 
actively engaged in learning. 



Creative activities: Young person spends three or more Religious community: Young person spends 
hours per week in lessons or practice in music, theatre, or one hour or more per week in activities in 
other arts. 



Bonding to school: Young person cares 
about her school. 



High expectations: Both parent(s) and teachers 
encourage the young person to do well. 

Youth as resources: Young people are given useful 
roles in the community. 

Family boundaries: Family has clear rules and 
consequences and monitors the young 
person's whereabouts. 

Caring school climate: School provides a caring, 
encouraging environment. 

Caring neighbourhood: Young person 
experiences caring neighbourhoods. 



a religious institution. 

Positive peer influence: Young person's best 
friends model responsible behaviour. 

Youth programs: Young person spends three 
or more hours per week in sports, clubs, or 
organizations at school and/or in the community 

Neighbourhood boundaries: Neighbours take 
responsibility for monitoring young 
person's behaviour. 

Community values youth: Young person 
perceives that adults in the community value 
youth. 

Time at home: Young person is out with 
friends "with nothing special to do" two 
or fewer nights per week. 



Caring: Young person places high value on 
helping others. 

Interpersonal competence: Young person 
has empathy, sensitivity and friendship skills. 

Personal power: Young person feels she had 
control over "things that happen to me". 

Responsibility: Young person accepts and 
takes personal responsibility. 

Self-esteem: Young person reports having 
high self-esteem. 



Positive view of personal future: Young 
person is optimistic about her 
personal future. 

Restraint: Young person believes it is 
important not to be sexually active or 
to use alcohol or other drugs. 

Achievement motivation: Young person 
is motivated to do well in school. 



Planning and decision making: Young 
person knows how to plan ahead and 
make choices. 

Reading for pleasure: Young person 
reads for pleasure three or more hours 
per week. 

Honesty: Young person "tells the truth 
even when it is not easy". 

Cultural competence: Young person has 
knowledge of and comfort with people of 
different cultural/racial /ethnic backgrounds. 

Sense of purpose: Young person reports 
that "my life has a purpose". 

Peaceful conflict resolution: Young 
person seeks to resolve conflict 
nonviolently. 

Resistance skills: Young person can 
resist negative peer pressure and 
dangerous situations. 
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is also research to suggest that females benefit more from 
competency enhancement or life skills training than do 
males, however, both derive some benefits [38]. Individual 
factors have also been shown to be specifically related to 
drug use and abuse for females in rural communities. For 
example, one study found that rural female youth who 
showed aggressive characteristics, who lacked parental 
support, and who were isolated from peer attachments 
should be the primary target of substance use prevention 
programs [40]. Research has also relayed that healthy 
identity and cultural connections are protective factors for 
Aboriginal youth and endemic to their well-being [41,42]. 
Although the 40 developmental assets have not been ap- 
plied extensively to at-risk Aboriginal youth, some work 
in Canada has highlighted their applicability [43] . 

In this paper, we share the 'preparatory' outcomes of 
our team's program evaluation of the Prince Albert Park- 
land Health Region Mental Health and Addiction Ser- 
vices' Outreach Worker Service (OWS) in eight rural, 
community schools three years following its implementa- 
tion. In the province of Saskatchewan, the Ministry of 
Health supports the application of empirically-based ap- 
proaches to youth drug prevention services to achieve its 
goal of "targeting the effective use of child and youth 
mental health and addictions resources toward improved 
mental health and well-being functioning for children, 
youth, and their families" (Saskatchewan Ministry of 
Health, Community Care Branch [44]). Despite this, and 
for a myriad of reasons (e.g., funding time constraints), it 
sometimes happens that provincially-funded programs are 
delivered prior to the establishment of evidence-informed 
goals and objectives. This is the case with the OWS. 

The paper begins with a brief overview of the OWS pro- 
gram. Next, we detail the establishment of the OWS's 
overarching program goals and objectives by our program 
evaluation alongside the OWS. We did this through in- 
depth focus groups with the OWS staff, OWS and school 
administrator interviews, and less intensive focus groups 
with school staff. With this understanding, we then detail 
how and why our evaluation team and the OWS staff 
aligned the program with the Positive Youth Development 
theoretical evidence-base, emphasizing the program's uni- 
versality, systems focus, strength base, and promotion of 
assets. This included gaining increased clarity about the 
OWS's definition of and engagement with its 'at-risk' stu- 
dent population. The paper concludes with a discussion 
of how, if time does not permit for the establishment of 
evidence-informed goals and objectives at the start-up of 
a program, obtaining insight and expertise from program 
personnel and school staff and administrators can bring 
the program to a point where this can be achieved and 
theoretical linkages made after a program has been 
implemented. This is a necessary foundation for measur- 
ing an intervention's success. 



Outreach Worker Service 

The OWS originated in 1996 in urban high schools in the 
Prince Albert Parkland Health Region (PAPHR) and was 
expanded in 2005 into one rural school. PAPHR is located 
in north central Saskatchewan, and is the third largest of 
Saskatchewan's health regions with a population of ap- 
proximately 80,000, with diversity most prominent by Abo- 
riginal student representation. Motivated by a provincial 
funding opportunity in 2007, Health Region administrators 
further expanded the OWS to six rural schools in two 
PAPHR school divisions in northeastern Saskatchewan that 
were informally identified as having 'at-risk' 12-17 year old 
youth populations. At-risk' was understood as meaning 
that youth were 'at risk' both for substance abuse problems 
and potential co-occurring mental health concerns. 

The environment of the communities in which the 
eight participating rural schools are situated have char- 
acteristics that could give rise to high levels of drug use. 
First, many of the communities experience increased 
levels of poverty. Many of the parents in these commu- 
nities work outside of the local community, which limits 
their parental and school involvement. Excessive alcohol 
use is highly normalized amongst the adult community 
populations. And there is a lack of extracurricular activ- 
ities available to youth, coupled with limited accessibility 
of the activities that exist due to transportation barriers. 

The initial expectations of the OWS were to screen, as- 
sess, and work with students on substance abuse and less 
severe mental health related issues (e.g., mild depression), 
act as a triage and referral unit for more severe mental 
health related issues (e.g., autism), and provide standard 
skills building programming (e.g., group counseling sup- 
port). According to senior PAPHR Administrators with 
clinical experience in rural schools, this set up two general 
expectations for the outreach worker positions: (1) they 
would be the sole resource for substance abuse/addiction 
related issues at a given school, and (2) they should at- 
tempt to build trusting relationships with all members of 
the school community for meaningful youth engagement. 

Although minimizing risk was the initial focus of the 
OWS, the role of protective factors was also acknowl- 
edged. This coincides with the Saskatchewan Ministry of 
Education designation of a community school, of which 
six of the PAPHR schools are officially designated. "Com- 
munity Schools are [to be] responsive, inclusive, culturally 
affirming and academically challenging. The learning pro- 
gram and environment effectively build on strengths to 
address the needs of the communities they serve" (Gov- 
ernment of Saskatchewan, Community Schools website). 
The Ministry promotes the community school philosophy 
across the province based on its universal value, while rec- 
ognizing that it is especially meaningful for 'at-risk' youth. 

The eight participating PAPHR schools varied in their ac- 
cess to OWS program staff based on perceived need, with 
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three schools accessing a full time employee, one school 
accessing one worker four days a week, and four schools 
accessing one worker approximately one day a week. All of 
the OWS staff held social work-related skills and compe- 
tencies to undertake the position. Each school worked in 
collaboration with the Health Region to set-up its OWS. 
On the ground, the outreach workers designed their pro- 
grams based on their school demographics, a strength- 
based philosophy (e.g., 40 developmental assets), and their 
individualized professional skill sets. All of the initial expec- 
tations of the OWS were addressed (i.e., screen, assess, 
work with students, referral, standards skills building), and 
added to during the three years of program implementa- 
tion: specialty skills-building programming (e.g., wilderness 
camping), (5) educational services (e.g., classroom educa- 
tion), (6) policy changes and development (e.g., disciplinary 
role), and (7) community development (e.g., inclusion of 
family in school activities). Limited attention was allotted 
throughout to the evidence-based theoretical literature. 

Methods 

Data collection and analysis for the preparatory stage of 
our team's program evaluation focused on: (1) 
establishing the overarching OWS program goals and 
objectives across the eight schools, (2) defining the 
targeted youth student population and OWS's engage- 
ment with them, (3) and identifying and then aligning 
the OWS with the empirically-informed Positive Youth 
Development (PYD) theoretical framework. Data collec- 
tion for all stages drew upon multiple sources, utilizing 
a mixed-methods approach. Others have found mixed 
methods to be ideally suited for all stages of program 
evaluation and specifically to evaluate drug abuse pro- 
gramming [45] . Ethics approval for this study was attained 
from the two participating PAPHR school districts. 

Data were collected in focus group and interview for- 
mats, including four in-depth focus groups with the 
OWS staff (averaged four 180 minutes each), three OWS 
interviews (two managers and a clinical supervisor) (ave- 
rage 90 minutes each) and nine school administrator in- 
terviews (principals, vice-principals and superintendents) 
(average 60 minutes each), and eight less intensive focus 
groups with informed school staff (average 90 minutes 
each; 8 principals, 2 superintendents, and 23 school staff, 
mainly teachers). Semi-structured interview and focus 
group guides were developed specific to each of the pop- 
ulations. The guides were structured to facilitate the col- 
lection of background information on the start-up of the 
OWS in each school, patterns of youth substance use, 
and the scope and nature of the drug prevention service 
offerings and needs. Data collected through a question- 
naire with OWS staff and analysis of existing OWS pro- 
gram data are not presented in this paper because of 
their separate focus. 



All in-depth focus groups and interviews were conducted 
by the project leads (CAD and CRD) and the less intensive 
focus groups with the school staff by the research assistants 
(MB, MS, HT). Notes from the interviews and focus groups 
were recorded electronically by a note taker, as close to ver- 
batim as possible. The data were analyzed by the project 
leads using a manual, general inductive approach. The cod- 
ing process involved recognizing important points/state- 
ments, from which categories were developed, and the 
findings subsequendy interpreted. Our choice of an induct- 
ive approach allowed the findings to emerge from the ob- 
jectives of the preparatory stage of our program evaluation. 
We undertook coding consistency checks through in- 
dependent parallel coding measures and applied stake- 
holder checks to our interpretations and findings with 
the OWS staff and administration. A general inductive 
approach was chosen because it "...provides an easily 
used and systematic set of procedures for analyzing 
qualitative data that can produce reliable and valid 
findings" [46] (237). We did not choose a more com- 
plex qualitative analytic approach (e.g., grounded the- 
ory, phenomenology, discourse analysis) because of the 
specific evaluation aim of the study [47-49]. 

Results 

Outreach Worker Service goals and objectives 

Applying our choice of methods, the overarching goals 
(see Outreach Worker Service goals) and objectives (see 
Outreach Worker Service objectives) of the OWS were 
established across the eight schools. Although at first it 
appeared that each school may have been unique in the 
service it offered because of varying programming activ- 
ities, the four in-depth focus groups held with the OWS 
staff, totaling more than 12 hours, relayed that this was 
not the case. Using a workbook designed to assist orga- 
nizations with no or little program evaluation experience 
as a guide (A Community-Based Workbook for Evaluat- 
ing Substance Abuse & Mental Health Programs in Sas- 
katchewan [50]), two well-defined goals and four 
corresponding objectives were developed for the OWS. 
They support the original and expanded expectations of 
the OWS during its three years of operation, and were 
verified in interviews with OWS and school administra- 
tors and focus groups with the school staff. 

Outreach Worker Service goals 

The goals of the OWS are: 

1. To improve the wellbeing of youth, schools and 
communities by increasing youths' 40 
developmental assets to prevent/minimize the 
problematic use of alcohol and other drugs and 
substances and related mental health concerns with 
four groups of youth through building therapeutic 
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relationships, asset development and application, 
education, and community development. 
2. To be part of the culture of the school by 
connecting with youth, families, the school and 
broader community. This includes community 
development, recognizing that communities within 
the Outreach Worker Service are at different stages 
of engagement. 

Outreach Worker Service objectives 

The objectives of the OWS are to: 

1. Build therapeutic relationships by 

i) engaging at risk students through an orientation 
led by the outreach worker 

ii) supporting the counseling of at-risk students by 
engaging caregivers when relevant 

iii) applying the appropriate therapeutic tools to 
increase student wellbeing 

iv) recognizing client needs and determining 
appropriate referral services as necessary 

v) collaborating with school staff on a student's 
therapeutic process as appropriate 

vi) providing support services to community 
members as requested (and as time permits) 

2. Develop and apply students' assets by 

i) developing and delivering life skills programming 
(i.e., 40 developmental assets) 

ii) providing opportunities to apply/practice life 
skills (experiential learning) 

3. Offer education by 
Youth 

i) educating student clients (which includes 
their families) in one-on-one mental health and 
substance abuse related issues, as identified 
through an individual assessment session 

ii) educating students in one-on-one and 
classroom group sessions 

iii) acting as an informational and educational 
resource to students and families 

Staff 

i) serving as a resource to staff to meet their 
health curriculum needs 

ii) serving as a resource to staff to meet student 
needs as identified by staff and/or outreach 
workers 

Community 

i) educating the community on promoting 
student health 

4. Develop community (it takes a whole village to raise 
a child philosophy) by 



i) Supporting community opportunities for access to 
and growth of services by promoting connectedness 
and well-being. 

'At-risk' youth 

Intrinsic to discussions during the in-depth OWS staff 
focus groups was a recurring and complex dialogue re- 
garding the program's target population, originally de- 
fined as youth between 12-17 years of age and 'at-risk' 
of substance use and dependence. There was variability 
amongst the OWS staff in defining 'at-risk' youth, which 
mirrors the discrepancy found in the research literature 
[51-53]. There was underlying agreement, however, 
about the circumstances that place some youth 'at 
greater risk' for problematic substance use, primarily in- 
dividual/personal, interpersonal (family, peers, school), 
and social and cultural/community factors. 

Three additional themes emerged from the OWS staff 
focus groups regarding the definition of youth as 'at- 
risk'. First, it was felt that there was divergence between 
the strength-based focus of the OWS and the deficit 
based focus of defining an 'at-risk' youth population. 
Second, the OWS staff were unwilling to place students 
in dichotomous categories of either 'at-risk' or 'not at- 
risk! and hence their preference for a tiered approach in- 
cluding 'low risk', 'moderate risk' and 'high risk' to use 
and abuse substances [51]. However, despite their desire 
to not use a dichotomous categorization, program staff 
indicated that such a classification was necessary for ser- 
vice delivery. For example, identifying students as 'high- 
risk' in an assessment helps to ensure that their services 
are prioritized. Moreover, mental health problems and 
issues concerning general well-being (e.g., youth having 
trouble at home, youth coping with parental break-up, 
peer issues like bullying, problems with self-image, 
among others) were identified as important markers for 
being 'at-risk' of substance use and dependence. Finally, 
the impacts of the social determinants of health (e.g., 
education, social support, gender, social economic sta- 
tus) were also recognized in the focus groups. 

It is important to note as well that the factors placing 
youth 'at-risk' and mentioned most prominently in the 
OWS staff focus groups were similarly identified in the 
semi-structured interviews with OWS and school admin- 
istrators and the focus groups with school staff. They 
highlighted their rural specificity, including social eco- 
nomic status (foremost poverty but also high economic 
status associated with rural economies due to farming in- 
come and access to funds to purchase substances), familial 
issues, such as experiences of violence and abuse and sin- 
gle parenting (including disruption with typically a father 
away at work for extended periods of time), younger kids 
associating with older kids (rural schools are K-12), and 
event-specific opportunities, such as rural bush parties. 
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One principal commented: "Certain events and time of 
the year have heavier use, especially of alcohol (e.g., sport- 
ing tournaments, graduation)... [There is] [sjome torrid 
history of this within the health region. Parental supervi- 
sion may be involved in approving alcohol use at an 
event" (School Principal, Interview #6). 

Based on this understanding, four 'at-risk' student groups 
were identified by the OWS workers and verified in the 
follow-up school administration interviews and school staff 
focus groups (see Outreach Worker Service 'At-risk' youth). 
The categories are hierarchically ranked, indicating the 
OWS staff concentrated their work based on identified 
need with the 'high-risk' students, followed by mental 
health, general prevention and then crisis situations. A 
school staff member succinctly shared their general 
insight and support for such a comprehensive defin- 
ition of 'at-risk' youth: "The four categories are specific 
but also broad, and they would include everyone, and 
no one would fall through the cracks" (School Staff, 
Interview #21). 

Outreach Worker Service 'At-risk' youth 

1. High risk youth who are not using substances (high 
risk meaning low on developmental assets). 

High risk youth using substances 

High risk youth abusing substances (e.g., binging 

on the weekend) 

2. Mental health issue related youth not using 
substances (e.g., depressed youth and used cannabis 
once, as an example; this category includes self- 
harm potential) 

Mental health issue related youth using substances 
Mental health issue related youth abusing 
substances 

Mental health issue related youth who specifically 
have FASD (may or may not be using/ abusing 
substances) 

3. Low risk at present youth (prevention focus with 
these youth) 

This category may also include pre-mental health 
issue related youth (for example, have low social/ 
life skills) 

Male and female students were identified by the OWS 
and school staff and administration as being at equal 
risk, with acknowledgement that they may face different 
risk factors. For example, patriarchal gender roles are 
well established in rural communities and can have a 
detrimental influence on girls' lives. Teen pregnancy was 
identified as an associated concern. Little to no attention 
was paid to other diversity factors, such as gender 



identity and sexual orientation. Some attention was paid 
to cultural diversity, and specifically to Aboriginal youth. 
Aboriginal youth were identified as a specific 'at-risk' 
population, with schools having a higher Aboriginal stu- 
dent population identifying a higher incidence of eco- 
nomic disadvantage, foster care involvement, issues with 
school attendance and transiency. It was also relayed 
that there is a general lack of integration between First 
Nation reserve and non-reserve students. All schools in- 
dicated an effort (although to varying degrees) to include 
Aboriginal worldviews and cultural practices in their 
curriculum, with those schools receiving funding as a 
community-designated school being required to provide 
Aboriginal contextualized curriculum and to be cultur- 
ally affirming. 

Positive youth development 

Drawing on expert knowledge is important for the devel- 
opment of youth drug prevention programming, but at- 
tention must also be paid to aligning and coordinating 
professional health care services with a theoretically in- 
formed evidence-base for evaluation purposes. It is im- 
portant for mental health and addictions programming to 
link with theoretical constructs as they are considered es- 
sential for measuring intervention success and conducting 
program evaluation studies [54]. Grounded in a strength 
based approach, the PYD framework was identified from 
the theoretical literature as a potential evidence-base for 
the OWS. A systematic comparison of the OWS with 
PYD revealed an amicable fit, noting their near identical 
recognition of universality, a systems focus, a strength 
base, and the promotion of assets. 

The OWS supports the universal element of PYD the- 
ory; that is, targeting all youth rather than only 'at-risk' 
youth, with the understanding that all youth can benefit 
from strengthened environments. The OWS targets all 
youth in the student population, in contrast to only a 
group of 'at-risk' or 'at high risk' youth. As relayed, the 
OWS identifies all youth on a continuum of risk for sub- 
stance use and related mental health concerns. For ex- 
ample, youth identified as being at 'low-risk' are targeted 
with early prevention efforts. This includes classroom 
education sessions, life skills programming to develop and 
strengthen assets, and having access to the OWS worker 
as an informational and educational resource. As a further 
example, all students (high and low risk) potentially be- 
nefit from the strengthened context of the community 
and resources available within the school because of the 
OWS's presence. Nevertheless, it should be noted that 
some high-risk classified youth do necessarily receive 
more specialized services (i.e., family counselling, crisis 
intervention, etc.). Also, diversity is acknowledged within 
the universal framework of PYD and the OWS, for ex- 
ample, it is recognized that some student populations, 
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such as male and female students as well as Aboriginal 
students, are affected by unique risk factors and therefore 
these must be addressed in response. 

A systems view is explicitly supported within the 
OWS through its focus on the environment and sup- 
ports surrounding youth, specifically the school and 
immediate community. PYD proposes that youth de- 
velopment is embedded within many integrated, inter- 
active contexts, including family, school, community, 
society, culture and history. The OWS is designed on a 
philosophy of increasing student health while simul- 
taneously increasing school and community well-being 
, as outlined in the OWS goals and objectives and pro- 
motion of the adage that "it takes a village to raise a 
child" [55]. Family is also clearly identified as a part of 
the system by the OWS, evident in the OWS's inclu- 
sion of family members in counselling sessions and 
staff efforts to build relationships with school, family 
and community members. Finally, the OWS recognizes 
that for the youth population it is serving, the systems- 
related determinants of health impact many students' 
well-being and 'at-risk' status, and therefore must also 
be recognized. 

PYD supports a strength-based framework that views 
youth as resources to be nurtured, encouraging the 
growth of inherent capacities for positive development 
and competence. Similarly espoused by OWS is the view 
that youths' inherent capacities for healthy well-being 
need to be fostered. A strength-based framework is expli- 
cit in the goals of the OWS to increase youth, school, and 
community well-being, in light of the fact that it is a pre- 
vention and intervention program for substance use and 
mental health concerns. In other words, the goal to pre- 
vent and/or minimize substance abuse is intertwined with 
expected improved general well-being. Thus, the OWS fo- 
cuses on increasing and strengthening positive attributes 
(developmental assets, resiliency), rather than eliminating 
or preventing negative attributes (substance use). This fits 
with the PYD framework's recommendation that negative 
outcomes can be reduced by strengthening positive assets. 

With the strength-based focus of the OWS, and staff 
awareness of the 40 developmental assets, this was the 
natural overlapping point between the OWS and PYD's 
promotion of assets. The merits of the 40 developmen- 
tal assets were identified foremost for their strength- 
based foundation and ability to concurrently address 
students (personal issues), families (family dynamics) 
and the community (community engagement and 
linking to services and supports). The OWS placed 
specific emphasis on three assets: self-esteem, devel- 
oping a sense of purpose/future, and role modeling. 
Although not consistently referred to as the 40 devel- 
opmental assets by the OWS, they are a foundation to 
the program's development. 



Discussion 

Adding a 'preparatory' stage to our team's program evalu- 
ation of the OWS enabled us to identify the goals and ob- 
jectives of the OWS and define its targeted 'at-risk' 
student youth population three years after the program's 
initial implementation. This is in line with how program- 
ming is established in the community at times, including 
Saskatchewan, often with insufficient time and resources 
to identify measurable program goals and objectives at the 
outset [56,57]. For example, a study by Carman (2007) 
shared that few organizations that deliver programming 
have either the time or funds required to carry out an 
evaluation study, no matter how important they believe it 
to be (e.g., to relay to funders their success to maintain 
programming funding) [58]. Our team's approach also en- 
abled us to identify a theoretically informed evidence-base 
for the OWS. This, alongside clearly defined goals, objec- 
tives and a target audience, is essential for the eventual 
undertaking of a complete program evaluation process. 
Work in this area, as illustrated in a case study by Larsen, 
Tax and Butock (2009), supports the important role of 
evaluation in moving organizations forward, including 
unifying their service delivery [57]. 

The OWS's definition of its goals and objectives is 
supported in the literature on effective substance use 
prevention in youth and schools. The goals and objec- 
tives generally coincide with the empirical literature in- 
dicating the effects of protective factors in reducing the 
impact of risk factors for substance use. The overarching 
OWS's goals and objectives reflect the protective factors 
of building community assets, strengthening individual 
resilience [12,16,25], forming connections to school [59] 
and forming positive relationships with others, including 
caring adults [24]. 

More specifically, the OWS's emphasis on improving 
well-being (goal 1) reflects the strength-based approach 
supported in the literature [60]. Including a focus on at- 
risk students is likewise supported in the literature, with 
specific positive outcomes identified for targeted inter- 
ventions with at-risk students [61]. 

Offering life skills programming to students is another 
key feature of the OWS (goal 1 & objective 2) that re- 
searchers have shown to improve outcomes [62]. The 
most effective services are those that use cognitive be- 
havioral programming, such as life skills development 
[61-63]. In addition, the use of experiential activities for 
students to apply and practice life skills in an interactive 
delivery method has been shown to reduce substance 
use [61] and has been identified as a best practice at- 
risk approach [62]. 

Inherent in the OWS's goals and objectives is a focus not 
only on students, but also families, schools and communi- 
ties, such as increasing well-being (goal 1), connecting with 
additional groups (goal 2), as well as offering education 
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(objective 3) and investing in community (objective 4). In 
the literature, such a holistic approach that includes mul- 
tiple factors and emphasizes building collaborative rela- 
tionships is supported as a best standard/practice for 
substance use prevention in youth [60] and for school- 
based programs [29]. 

The expanded definition of 'at-risk' youth by the 
OWS is likewise supported in the current literature, 
where definitions of youth 'at-risk' for substance use 
vary. Indeed, some researchers apply the term 'at-risk' to 
include students who are at "low risk", "moderate risk" 
and "high risk" to abuse substances [64], although the 
emerging trend is to recognize student diversity and not 
categorize all students as one and the same [5,29,53,65]. 

The risk factors identified by the OWS correspond in 
large part to the common overriding risk factors 
discussed by previous researchers (e.g., negative risk tak- 
ing tendencies [2], poor psychological health [2], limited 
school and community bonding [15,16]). It is also im- 
portant to note that the OWS's recognition of the social 
determinants of health in combination with risk factors 
for youth substance abuse helps to more precisely de- 
fine, understand and respond to who 'at-risk' youth are. 

The loose theoretical linkage in developing the OWS to 
the 40 developmental assets framework amongst the ma- 
jority of the school sites is a key strength in its delivery of 
effective programming. Generally, when a program is 
based on an explicit theory and it is understood and used 
to design the activities of the program, it results in a more 
positive reduction of substance use or abuse [61]. This be- 
came even more explicit with alignment of the OWS to 
the Positive Youth Development theoretical evidence- 
base, emphasizing the program's universality, systems 
focus, strength base, and asset promotion. 

According to the Health Canada (2001) compendium of 
best practice guidelines for preventing substance use 
problems among young people, there are numerous char- 
acteristics of effective programming and they consistently 
emphasize a strength-based approach [60]. These include 
promoting youths' strength of resiliency, using a holistic 
approach (systems approach, including individual, school, 
family, and community factors), incorporating skill devel- 
opment as a main component, and intensifying the pro- 
gram as the risk of the youth increases. Health Canada's 
best practice guide suggests that as youth experience 
more risk factors, they are more likely to use substances, 
as well as engage in other problematic behaviours. The 
strengths of the youth are in a constant interaction with 
their risk factors. It follows that if protective factors can 
be strengthened, engagement with risky behaviours may 
decrease [60]. 

The focus on protective factors and resiliency along- 
side risk factors (a strength-based approach) is identified 
as a best strategy for positive behavioural and attitudinal 



outcomes [29,60]. Focusing on youths' capabilities and 
strengths rather than their limitations or deficits, in- 
fluences youth to respond to programs in a more posi- 
tive way. With this approach youth are active agents 
in their development, which helps to further foster 
protective factors. Some of the key protective factors 
that the 2010 Canadian School Standards identify are 
competence, self-confidence, connectedness, character, 
caring, compassion, and safe and welcoming environ- 
ments [29]. This strength-based approach, alongside 
the 2001 Health Canada best practice guide, coincide 
with the OWS's application of the 40 developmental 
assets and more specifically its adherence with the 
Positive Youth Development framework; both focus on 
positive youth development as opposed to being prob- 
lem or deficit-based. 

Developmental psychology researchers offer evidence for 
the effects of assets and PYD components in promoting 
healthy adolescent adjustment. It has been demonstrated 
that the presence of a high number of developmental as- 
sets is related to low levels of substance use [35]. An ex- 
ample of a PYD program designed to prevent problematic 
substance use is the Positive Youth Development Collab- 
orative, which teaches youth prevention skills and health 
education through culturally relevant activities. The main 
component of this prevention program focuses on effective 
decision making, coping with stress, information about 
substances, and applying decision making to one's own 
life. Results of an evaluation with mainly African Ameri- 
can and Hispanic participants in the United States indi- 
cated that students who received the intervention were 
more than six times as less likely to use alcohol than the 
control students, and they were also less likely to use 
marijuana as compared to the control students. The stu- 
dents in the intervention were also less likely one year 
after the intervention to use alcohol, marijuana, or other 
drugs than the control students. Although there was still 
an increase in substance use as the intervention partici- 
pants aged, it was significantly less than the increase with 
the control students [30] . 

Upon reflection, the OWS staff and administrations, 
with the guidance of our independent evaluation team, 
identified three key insights through their participation in 
the 'preparatory' stage of a program evaluation. First, they 
came to a clear understanding that it was necessary to 
undergo this preparatory stage to eventually conduct a 
program evaluation of the OWS, to which they are com- 
mitted. They shared how, in future program development, 
they will also be able to transfer this knowledge and its 
importance. Second, the staff recognized how similar they 
are in the OWS programming they offer, even though 
they are at different school locations and with somewhat 
varying populations. Identifying the overarching program 
goals and objectives made this clear. Similarly, developing 
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a definition of the 'at-risk' student population relayed how 
similar they are implementing the OWS program. And 
third, the OWS staff were able to clearly identify how they 
expanded the OWS program expectations since its start- 
up, as originally identified by the OWS administration. In 
fact, the absence of overarching goals and objectives may 
have allowed the program greater flexibility in its contin- 
ued development and response to student-identified 
needs. It was important to document this for future evalu- 
ation purposes, as well as a reminder to the OWS staff to 
better information share amongst themselves and with 
their administration. 

Limitations to the current study 

There are two key limitations to the current study. The 
first is that a 'comprehensive' program evaluation could 
not be completed because of the necessary upfront work 
that needed to be undertaken, including identifying the 
OWS goals and objectives, defining 'at-risk' youth, and 
linking the OWS to an empirically-informed theoretical 
framework, which in this case was PYD. The next step 
for the OWS will be to undertake a process evaluation 
with our evaluation team, commonly defined as "[t]he 
systematic collection of information on a program's in- 
puts, activities, and outputs, as well as the program's 
context and other key characteristics" [66]. The evalu- 
ation work completed to date will be beneficial to ad- 
dressing the overall goal of a process evaluation, which 
is to document what is happening in a program and rec- 
ognizing, as noted in a tobacco control process evalu- 
ation, that "the type of information gathered, and its 
frequency, will depend on the kinds of questions that 
you seek to answer" [66]. The outcome of the process 
evaluation in our work is an important step forward in 
establishing the groundwork to link progress and activ- 
ities to outcomes in a future outcome evaluation. 

Second, the data collected in this study is not ge- 
neralizable beyond the PAPHR in Saskatchewan, Canada. 
However, the documentation of the research process and 
content is a significant contribution to the literature and 
to others undertaking program evaluations; it is necessary 
to document the difficulties and solutions to doing re- 
search in the real- world [56]. Similarly, Dykeman et al. 
(2003) wrote an academic article on the development of a 
program logic model to measure the processes and out- 
comes of a program "because we found limited descrip- 
tions of the process in the literature" (197) [67]. 

Conclusions 

This paper shared the 'preparatory' outcomes of our team's 
process evaluation of the Prince Albert Parkland Health 
Region Mental Health and Addiction Services' Outreach 
Worker Service (OWS) in eight rural, community schools 
three years following its implementation. It was necessary 



to begin at this preparatory stage because as sometimes 
happens, programming was delivered prior to the establish- 
ment of evidence-informed goals and objectives. Our team 
helped to establish the overarching program goals and ob- 
jectives, and its targeted definition and engagement with an 
'at-risk' student population. This, in turn, enabled us to 
align the OWS with the empirically-informed Positive 
Youth Development (PYD) theoretical framework, acknow- 
ledging the overlap between it and the OWS's attention to 
developmental assets. Through this process we concluded 
that undertaking a process evaluation three years after the 
OWS's initial implementation was possible. In fact, this 
time frame may have allowed the program greater flexibility 
in its continued development and response to student- 
identified needs. Nonetheless, this is a necessary foundation 
for measuring an intervention's success. This is an import- 
ant contribution to the literature and for others trying to 
undertake programming research in the 'real world'. 
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